Coherence Research of Audio-Visual Cross-Modal Based on HHT
نویسندگان
چکیده
Visual and aural modes are two main manners that human utilize to senses the world. Their relationship is investigated in this work. EEG experiments involving mixed aural and visual modes are designed, utilizing HilbertHuang Transform (HHT) and electroencephalogram (EEG) signal processing techniques. During EEG data processing, I-EEMD method of similar weighted average waveform extension is proposed to decompose the EEG signals, specifically accounting for the problem of end effects and mode mixing existing in the traditional HHT. The main components of are obtained after decomposing the signals including mixed modes with I-EEMD respectively. The correlation coefficient of consistent and inconsistent mixed signal is calculated, and the comparison is made. Investigation on the comparison condition of the correlation coefficient indicates that there is coherence in both the visual and aural modes.
منابع مشابه
Development of a Bayesian Estimator for Audio-Visual Integration: A Neurocomputational Study
The brain integrates information from different sensory modalities to generate a coherent and accurate percept of external events. Several experimental studies suggest that this integration follows the principle of Bayesian estimate. However, the neural mechanisms responsible for this behavior, and its development in a multisensory environment, are still insufficiently understood. We recently p...
متن کاملAuditory selective attention is enhanced by a task-irrelevant temporally coherent visual stimulus in human listeners
In noisy settings, listening is aided by correlated dynamic visual cues gleaned from a talker's face-an improvement often attributed to visually reinforced linguistic information. In this study, we aimed to test the effect of audio-visual temporal coherence alone on selective listening, free of linguistic confounds. We presented listeners with competing auditory streams whose amplitude varied i...
متن کاملAuditory cross-modal reorganization in cochlear implant users indicates audio-visual integration
There is clear evidence for cross-modal cortical reorganization in the auditory system of post-lingually deafened cochlear implant (CI) users. A recent report suggests that moderate sensori-neural hearing loss is already sufficient to initiate corresponding cortical changes. To what extend these changes are deprivation-induced or related to sensory recovery is still debated. Moreover, the influ...
متن کاملE-skin: Research into Wearable Interfaces, Cross-modal Perception and Communication for the Visually Impaired on the Mediated Stage
Today our cultural events are dominated by visual information based on sight and sound, but hardly at all on the combined senses of touch and sound. Visually impaired users, lacking sight are not often able to engage in cultural events. Compensated audio-description or talking books are standard products that visually impaired people can buy to imagine feature film stories or decide what is hap...
متن کاملCross-modal contextual coherence of information integration in people with Williams syndrome.
This study aimed to explore the generalization of contextual integration from within-modality (visual-visual) to cross-modal (visual-auditory) processing in people with Williams syndrome (WS), and to clarify whether the concreteness or social relatedness of stimuli contributed to contextual coherence using pictures. Contextual coherence was evaluated in accordance with context-appropriateness b...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- Journal of Multimedia
دوره 9 شماره
صفحات -
تاریخ انتشار 2014